{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## MLP network for MNIST digits classification\n",
    "In this example, we demonstrate how to build, train, and validate a model that classifies MNIST images.\n",
    "Let us load the dataset first."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "from __future__ import absolute_import\n",
    "from __future__ import division\n",
    "from __future__ import print_function\n",
    "\n",
    "import numpy as np\n",
    "import tensorflow\n",
    "from tensorflow.keras.models import Sequential\n",
    "from tensorflow.keras.layers import Dense\n",
    "from tensorflow.keras.utils import to_categorical, plot_model\n",
    "from tensorflow.keras.datasets import mnist\n",
    "\n",
    "# load mnist dataset\n",
    "(x_train, y_train), (x_test, y_test) = mnist.load_data()\n",
    "\n",
    "# compute the number of labels\n",
    "num_labels = len(np.unique(y_train))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Pre-processing of data\n",
    "\n",
    "The raw dataset is not ideal for training a machine learning model. The images are still in raw grayscale format wherein each pixel is $\\epsilon [0,255]$. The labels are in raw digit format $\\epsilon [0,9]$. Both input and output data must be pre-processed first.\n",
    "\n",
    "### Convert raw labels into one-hot vector\n",
    "The labels are in raw digit format (e.g. 0,1, ...,9). It is not ideal for computing the $y=f(x)=p(y|x)$ which is a mulitnoulli or categorical distribution. Therefore, the labels are converted into one-hot vector using `tf.keras` `to_categorical()` utility function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "# convert to one-hot vector\n",
    "# e.g. 3 -> [0 0 0 1 0 0 0 0 0 0]\n",
    "y_train = to_categorical(y_train)\n",
    "y_test = to_categorical(y_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Normalizing the pixel values\n",
    "The pixel values (e.g. 0 to 255) is normalized to $[0.0, 1.0]$ so that the network can be easily trained. In other words, no wild fluctuations in weights in biases."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "# image dimensions (assumed square)\n",
    "image_size = x_train.shape[1]\n",
    "input_size = image_size * image_size\n",
    "\n",
    "# resize and normalize\n",
    "x_train = np.reshape(x_train, [-1, input_size])\n",
    "x_train = x_train.astype('float32') / 255\n",
    "x_test = np.reshape(x_test, [-1, input_size])\n",
    "x_test = x_test.astype('float32') / 255"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Hyper-parameters\n",
    "A typical network has many hyper-parameters that we can tune.\n",
    "\n",
    "`batch_size` is the number of samples used per SGD. There is no right value of `batch_size`. Usually, it ranges in the multiples of 2 (e.g 2, 4, 8, 16, 32, 64, 128, 256). The lower the `batch_size`, the more steps in SGD. Also, the less accurate is the estimate of the true gradient. The higher the value of `batch_size`, the less number of steps in SGD. The network might not be completely trained after the number of epochs indicated. High value leads to a better gradient estimate. This tradeoff let us choose a value of `128`. \n",
    "\n",
    "When processing huge data like big image sizes, the batch size is limited by the GPU memory of your PC.\n",
    "\n",
    "One `epoch` means we have completed one training using the entire dataset. An `epoch` is equal to `len(x_train)/batch_size`.\n",
    "\n",
    "`n_units` is simply the number of units of each MLP layer. For this example, we use a uniform number. However, this can have different numbers. In encoders, this number usually decreases with depth. In decoders, it is the reverse."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "# hyper-parameters\n",
    "batch_size = 128\n",
    "n_units = 256\n",
    "epochs = 20"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Model building\n",
    "Let us build a 3-layer MLP. Each layer is a `Dense()` with `relu` activation. \n",
    "\n",
    "Since this is classifier, the last layer has number of units equal to the number of digit categories. The last layer uses a `softmax` activation to convert the prediction into probabilities. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model: \"3-layer-MLP\"\n",
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "input_layer (Dense)          (None, 256)               200960    \n",
      "_________________________________________________________________\n",
      "hidden_layer (Dense)         (None, 256)               65792     \n",
      "_________________________________________________________________\n",
      "output_layer (Dense)         (None, 10)                2570      \n",
      "=================================================================\n",
      "Total params: 269,322\n",
      "Trainable params: 269,322\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n"
     ]
    }
   ],
   "source": [
    "# model is a 3-layer MLP\n",
    "model = Sequential(name='3-layer-MLP')\n",
    "model.add(Dense(n_units, input_dim=input_size, activation='relu', name='input_layer'))\n",
    "model.add(Dense(n_units, activation='relu', name='hidden_layer'))\n",
    "model.add(Dense(num_labels, activation='softmax', name='output_layer'))\n",
    "model.summary()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Training and evaluation (testing)\n",
    "\n",
    "Let's train the model using SGD and let's monitor one important metric in classification - `accuracy`. At every epoch, we conduct a validation in order to see if there is an improvement in test accuracy."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/20\n",
      "469/469 [==============================] - 1s 3ms/step - loss: 1.2387 - accuracy: 0.6988 - val_loss: 0.5954 - val_accuracy: 0.8560\n",
      "Epoch 2/20\n",
      "469/469 [==============================] - 1s 3ms/step - loss: 0.4945 - accuracy: 0.8727 - val_loss: 0.4006 - val_accuracy: 0.8911\n",
      "Epoch 3/20\n",
      "469/469 [==============================] - 1s 2ms/step - loss: 0.3845 - accuracy: 0.8954 - val_loss: 0.3421 - val_accuracy: 0.9025\n",
      "Epoch 4/20\n",
      "469/469 [==============================] - 1s 3ms/step - loss: 0.3393 - accuracy: 0.9048 - val_loss: 0.3098 - val_accuracy: 0.9118\n",
      "Epoch 5/20\n",
      "469/469 [==============================] - 1s 2ms/step - loss: 0.3112 - accuracy: 0.9121 - val_loss: 0.2879 - val_accuracy: 0.9179\n",
      "Epoch 6/20\n",
      "469/469 [==============================] - 1s 2ms/step - loss: 0.2906 - accuracy: 0.9173 - val_loss: 0.2721 - val_accuracy: 0.9216\n",
      "Epoch 7/20\n",
      "469/469 [==============================] - 1s 2ms/step - loss: 0.2741 - accuracy: 0.9221 - val_loss: 0.2577 - val_accuracy: 0.9255\n",
      "Epoch 8/20\n",
      "469/469 [==============================] - 1s 2ms/step - loss: 0.2598 - accuracy: 0.9260 - val_loss: 0.2470 - val_accuracy: 0.9296\n",
      "Epoch 9/20\n",
      "469/469 [==============================] - 1s 3ms/step - loss: 0.2471 - accuracy: 0.9294 - val_loss: 0.2330 - val_accuracy: 0.9331\n",
      "Epoch 10/20\n",
      "469/469 [==============================] - 1s 3ms/step - loss: 0.2359 - accuracy: 0.9327 - val_loss: 0.2249 - val_accuracy: 0.9348\n",
      "Epoch 11/20\n",
      "469/469 [==============================] - 1s 2ms/step - loss: 0.2255 - accuracy: 0.9357 - val_loss: 0.2156 - val_accuracy: 0.9365\n",
      "Epoch 12/20\n",
      "469/469 [==============================] - 1s 2ms/step - loss: 0.2161 - accuracy: 0.9387 - val_loss: 0.2072 - val_accuracy: 0.9395\n",
      "Epoch 13/20\n",
      "469/469 [==============================] - 1s 3ms/step - loss: 0.2075 - accuracy: 0.9410 - val_loss: 0.1998 - val_accuracy: 0.9413\n",
      "Epoch 14/20\n",
      "469/469 [==============================] - 1s 3ms/step - loss: 0.1994 - accuracy: 0.9434 - val_loss: 0.1917 - val_accuracy: 0.9438\n",
      "Epoch 15/20\n",
      "469/469 [==============================] - 1s 2ms/step - loss: 0.1918 - accuracy: 0.9454 - val_loss: 0.1854 - val_accuracy: 0.9455\n",
      "Epoch 16/20\n",
      "469/469 [==============================] - 1s 2ms/step - loss: 0.1846 - accuracy: 0.9477 - val_loss: 0.1801 - val_accuracy: 0.9464\n",
      "Epoch 17/20\n",
      "469/469 [==============================] - 1s 2ms/step - loss: 0.1783 - accuracy: 0.9494 - val_loss: 0.1739 - val_accuracy: 0.9489\n",
      "Epoch 18/20\n",
      "469/469 [==============================] - 1s 2ms/step - loss: 0.1721 - accuracy: 0.9508 - val_loss: 0.1698 - val_accuracy: 0.9491\n",
      "Epoch 19/20\n",
      "469/469 [==============================] - 1s 3ms/step - loss: 0.1663 - accuracy: 0.9527 - val_loss: 0.1636 - val_accuracy: 0.9517\n",
      "Epoch 20/20\n",
      "469/469 [==============================] - 1s 3ms/step - loss: 0.1609 - accuracy: 0.9541 - val_loss: 0.1588 - val_accuracy: 0.9532\n",
      "\n",
      "Test accuracy: 95.3%\n"
     ]
    }
   ],
   "source": [
    "# loss function for one-hot vector\n",
    "# use of sgd optimizer with default lr=0.01\n",
    "# accuracy is a good metric for classification tasks\n",
    "model.compile(loss='categorical_crossentropy',\n",
    "              optimizer='sgd',\n",
    "              metrics=['accuracy'])\n",
    "# train the network\n",
    "history = model.fit(x_train, y_train, validation_data=(x_test, y_test), epochs=epochs, batch_size=batch_size)\n",
    "\n",
    "# validate the model on test dataset to determine generalization\n",
    "loss, acc = model.evaluate(x_test,\n",
    "                           y_test, \n",
    "                           batch_size=batch_size,\n",
    "                           verbose=False)\n",
    "print(\"\\nTest accuracy: %.1f%%\" % (100.0 * acc))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Examine the train history\n",
    "\n",
    "Let's examine how the loss (cross-entropy) improves with epochs. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEWCAYAAABrDZDcAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3dd3wc1bnw8d8jaaVdrcpKllwk2cjYphhjGyOKgWCcGGIIN3B5Q4BLCzWEQCDtxu+bRrv3EvJJo+NQb0gwpEAcQicQkgAGGWzjgrFxlZt679Lz/jEjeS2vpLWt3ZU1z/fzmc/OzpyZfbRe77NnzpxzRFUxxhjjXUmJDsAYY0xiWSIwxhiPs0RgjDEeZ4nAGGM8zhKBMcZ4nCUCY4zxOEsE5qAjIski0igiE4ZBLP8Uka8kOg5jDoQlAhNz7pd2z9ItIi1hzy/e1/OpapeqZqjqlljEOxRE5OGwv7FdRDrCnv/lAM57g4i8NEiZUhG5cH9fw3hPSqIDMCOfqmb0rIvIJuBqVX2tv/IikqKqnfGILVZU9WrgagARuQMoUtWvJDQoY/phNQKTcCJyh4g8LSJPiUgDcImIzBaRd0WkVkR2iMjdIuJzy6eIiIpIsfv8SXf/iyLSICLviMjEfl4rSUT+ICI73XO/KSJHhu0f8FwiMl9E1opInYj8CpAD+LvniMh7bhxLRWR22L7rRGSzG8OnInKeiBwH/AyY59YsyvbjNb8sImtEpEZEXhWRSWH7bnHf63q3zEnu9s+IyDJ3+w43sZkRxBKBGS7+HfgdkA08DXQCNwF5wMnAfOCrAxz/H8APgVxgC3D7AGWfB6YAY4GVwG+iOZeIjAb+ACxw4yoDTojy79uD+wX8J/dcucCtwJ9FJFtE8oH/BuaqaiZwKrBaVd8Hvg285l4aK9rH15wJPILzPo4B3nZfM1lEjgUuBqbj/BucDWxzD70fuFVVs4DDgcX78zeb4csSgRku/qmqf1HVblVtUdX3VXWJqnaq6gZgITBngOP/oKqlqtoB/BaYGamQe/7HVbVBVVuBW4BjRSQYxbnOBpap6rPuvp8BFfv5934FeEZV/+bGtBhYB8wDunFqGkeJSJqqblPVj/fzdcJdBPxeVd9S1XacBFcAHIOTeNOBo4BkVf1UVTe7x3UAh4lIrqrWq+p7QxCLGUYsEZjhYmv4ExE5QkT+6l7CqQduw/kV3p+dYevNQEakQu6v37tEZIN73vXurvBz93eugvA4VbUbp1awPw4BvuJeFqoVkVqchFOgqlXA5cA3gV0i8ufwSzgHoADo+XLHbYfZDhSq6nLgB8D/AOUi8hu3ZgJwKVACrHMv150+BLGYYcQSgRku+g6D+xDOZZvJ7iWJH3EA1+PDXAacBXwW5xLIZHd7NOfeAYzveSIiScA+XZ4JsxV4UFVDYUtQVe8BUNXFqvpZoBDny/pe97gDGS54O04CApy2FpzksM19zcdUdTYwCcjESb6o6ipVPR8YDTwA/Mk91owQlgjMcJUJ1AFNbmPuQO0D+3reNqAK51LIf+3Dsc8DM0XkHPeL8JtA/iDH9Odx4CIRmes2YAdEZJ6IjBGR8SJylogEgFagCehyj9sFTIjii9gnIv6wJQVYBHxJRE52G96/j5PcPhSRaSJyqoik4dSCWnteU0Qucy8LdeH8m3RzYAnJDDOWCMxw9W2cyyMNOLWDp4fovI/h/DLeDqzCaTCNiqruAi4AfoqTSCYAS/YnCFVdB5wP3OGeaxNwI07NJAXnS3oXUIlzyegm99AXcH7BV4jIZvr3v0BL2HKvqn6Ik1AfwWnbOBU41/2CDwC/cGPZAaThtJ8AnAN84t7RdStwgXuMGSHEJqYxxhhvsxqBMcZ4nCUCY4zxOEsExhjjcZYIjDHG4w66e4Hz8vK0uLg40WEYY8xBZenSpZWqGvF254MuERQXF1NaWproMIwx5qAy0O3GdmnIGGM8zhKBMcZ4nCUCY4zxuIOujcAYM/J0dHRQVlZGa2trokM56Pn9foqKivD5fFEfY4nAGJNwZWVlZGZmUlxcjMhQDDLrTapKVVUVZWVlTJwYcZK+iOzSkDEm4VpbWxk1apQlgQMkIowaNWqfa1aWCIwxw4IlgaGxP++jZxLBxzvrueulj6lr7kh0KMYYM6x4JhFsqWrm/jc/ZUt1c6JDMcYMM1VVVcycOZOZM2cyduxYCgsLe5+3t7dHdY4rrriCtWvXRv2aDz/8MDfffPP+hjykPNNYXBAKALCttoWji7ITHI0xZjgZNWoUy5YtA+CWW24hIyOD73znO3uUUVVUlaSkyL+fH3vssZjHGSueqREUhiUCY4yJxvr165k2bRrXXXcds2bNYseOHVx77bWUlJRw1FFHcdttt/WWPeWUU1i2bBmdnZ2EQiEWLFjAjBkzmD17NuXl5QO+zsaNG5k7dy7Tp0/n9NNPp6ysDIBFixYxbdo0ZsyYwdy5cwH46KOPOO6445g5cybTp09nw4YNB/x3eqZGEEr3EfAls90SgTHD2q1/WcXq7fVDes6pBVn8+N+O2q9jV69ezWOPPcaDDz4IwJ133klubi6dnZ3MnTuXL33pS0ydOnWPY+rq6pgzZw533nkn3/rWt3j00UdZsGBBv69x/fXXc/XVV3PxxRezcOFCbr75Zv7whz9w66238uabbzJmzBhqa2sBuP/++/nOd77DBRdcQFtbG0Mxy2TMagQi8qiIlIvIyn72XywiK9zlbRGZEatY3NejIOS3RGCM2SeTJk3iuOOO633+1FNPMWvWLGbNmsWaNWtYvXr1XscEAgHOPPNMAI499lg2bdo04GssWbKECy+8EIDLLruMf/zjHwCcfPLJXHbZZTz88MN0d3cDcNJJJ3HHHXdw1113sXXrVvx+/wH/jbGsETwO3IsziXYkG4E5qlojImcCC4ETYhgPBaGAJQJjhrn9/eUeK8FgsHd93bp1/OpXv+K9994jFApxySWXRLxnPzU1tXc9OTmZzs7O/XrtX//61yxZsoTnn3+eGTNmsGLFCi699FJmz57NX//6V04//XSeeOIJTj311P06f4+Y1QhU9S2geoD9b6tqjfv0XaAoVrH0KMoJsK3WurAbY/ZPfX09mZmZZGVlsWPHDl5++eUhOe+JJ57IM888A8CTTz7Z+8W+YcMGTjzxRG6//XZycnLYtm0bGzZsYPLkydx000184QtfYMWKFQf8+sOljeAq4MX+dorItcC1ABMmTNjvFynIDlDZ2EZrRxd+X/J+n8cY402zZs1i6tSpTJs2jUMPPZSTTz55SM577733ctVVV/E///M/jBkzpvcOpG9+85ts3LgRVeWMM85g2rRp3HHHHTz11FP4fD4KCgq44447Dvj1ZSgaGvo9uUgx8LyqThugzFzgfuAUVa0a7JwlJSW6vxPT/HFpGd/+/XLe/M5pFOcFBz/AGBMXa9as4cgjj0x0GCNGpPdTRJaqakmk8gm9fVREpgMPA+dEkwQOVE9fAmsnMMaY3RKWCERkAvAn4FJV/SQer2l9CYwxZm8xayMQkaeA04A8ESkDfgz4AFT1QeBHwCjgfneQpM7+qi1DZUx2GiKWCIwxJlzMEoGqXjTI/quBq2P1+pGkpSSTn5Fml4aMMSaMZ4aY6OH0JbBbSI0xpofnEkGhdSozxpg9eC4RFIT8bKttGZLxOYwxI8NQDEMN8Oijj7Jz586I+y655BKee+65oQp5SA2XDmVxUxgK0NbZTXVTO6My0hIdjjFmGIhmGOpoPProo8yaNYuxY8cOdYgx5cEaQU9fAmsnMMYM7oknnuD4449n5syZXH/99XR3d9PZ2cmll17K0UcfzbRp07j77rt5+umnWbZsGRdccMGgNYlXX32VmTNncvTRR3PNNdf0lv3ud7/L1KlTmT59Ot/73veAyENRDzXP1QhsghpjhrkXF8DOj4b2nGOPhjPv3OfDVq5cybPPPsvbb79NSkoK1157LYsWLWLSpElUVlby0UdOnLW1tYRCIe655x7uvfdeZs6c2e85m5ubufLKK3nzzTeZNGlS79DT559/Pi+88AKrVq1CRHqHnY40FPVQ81yNoNB6FxtjovTaa6/x/vvvU1JSwsyZM/n73//Op59+yuTJk1m7di033XQTL7/8MtnZ0f+oXLNmDVOmTGHSpEmAM+z0W2+9RW5uLklJSVxzzTU8++yzvaOeRhqKeqh5rkZgE9QYM8ztxy/3WFFVrrzySm6//fa99q1YsYIXX3yRu+++mz/+8Y8sXLgw6nNG4vP5KC0t5dVXX2XRokU88MADvPLKKxGHos7JyTmgv6svz9UIeiaosd7FxpjBzJs3j2eeeYbKykrAubtoy5YtVFRUoKqcf/753HrrrXzwwQcAZGZm0tDQMOA5p06dyrp163qnmHzyySeZM2cODQ0N1NfXc/bZZ/OLX/yCDz/8EIg8FPVQ81yNAGyCGmNMdI4++mh+/OMfM2/ePLq7u/H5fDz44IMkJydz1VVXoaqICD/5yU8AuOKKK7j66qsJBAK89957e0xQ0yM9PZ1HHnmE8847j66uLk444QSuueYaysvLOe+882hra6O7u5uf//znQOShqIdaTIehjoUDGYa6x4I/ruC1NeWU/mDeEEVljDkQNgz10DqohqFOlILQ7glqjDHG6zyZCHruHNpZZ30JjDHGk4nAJqgxZvg52C5TD1f78z56MhHYBDXGDC9+v5+qqipLBgdIVamqqsLv9+/TcZ68a6hnghobZsKY4aGoqIiysjIqKioSHcpBz+/3U1RUtE/HeDIR2AQ1xgwvPp+PiRMnJjoMz/LkpSFw+xLUWSIwxhjPJoLCUIBtNZYIjDHGs4nAJqgxxhiHhxPB7glqjDHGyzydCMDuHDLGGM8mAutLYIwxDs8nAruF1BjjdZ5NBDZBjTHGODybCHomqLG+BMYYr/NsIgCnwXibNRYbYzzO04nAOpUZY0wME4GIPCoi5SKysp/9IiJ3i8h6EVkhIrNiFUt/bIIaY4yJbY3gcWD+APvPBKa4y7XAAzGMJaICm6DGGGNilwhU9S2geoAi5wD/q453gZCIjItVPJEUhJwxu+3OIWOMlyWyjaAQ2Br2vMzdthcRuVZESkWkdCjHK7dOZcYYk9hEIBG2RRwBTlUXqmqJqpbk5+cPWQBjs/02QY0xxvMSmQjKgPFhz4uA7fEMwCaoMcaYxCaCxcBl7t1DJwJ1qroj3kHYBDXGGK+L2VSVIvIUcBqQJyJlwI8BH4CqPgi8AJwFrAeagStiFctACkMB1uysT8RLG2PMsBCzRKCqFw2yX4Gvx+r1o1UQ8vP6x7tQVUQiNVsYY8zI5umexeBcGmrtsAlqjDHeZYnAJqgxxnic5xOB9SUwxnid5xNBgU1QY4zxOM8ngpx0H35fkiUCY4xneT4RiAiF1pfAGONhnk8EYBPUGGO8zRIBToOxXRoyxniVJQKcGkFFQxttnTZBjTHGeywRsPvOoR12ecgY40GWCLAJaowx3maJAOtUZozxNksEOBPUgA0zYYzxJksEuBPUZNoENcYYb7JE4LIJaowxXmWJwFUUClgbgTHGkywRuApCfrbXtuDMl2OMMd4RVSIQkRtEJCfWwSRSzwQ1Nc0diQ7FGGPiKtoawVjgfRF5RkTmywic09GGozbGeFVUiUBVfwBMAR4BvgKsE5H/FpFJMYwtrnr6EpTVWCIwxnhL1G0E7mTzO92lE8gB/iAid8UotriyGoExxqtSoikkIt8ALgcqgYeB76pqh4gkAeuA/4xdiPFhE9QYY7wqqkQA5AHnqerm8I2q2i0iZw99WPEnItaXwBjjSVElAlX9kYjMEpFzAAX+paofuPvWxDLAeCq0CWqMMR4U7e2jPwSeAEbh1A4eE5EfxDKwRCjItglqjDHeE+2lof8AjlHVVgARuRP4ALgjVoElQmHO7glq0lKSEx2OMcbERbR3DW0C/GHP04BPhzyaBOu5c2hnnV0eMsZ4R7SJoA1YJSKPi8hjwEqgUUTuFpG7+zvI7Xy2VkTWi8iCCPsniMgbIvKhiKwQkbP2788YGj0T1NiYQ8YYL4n20tCz7tLjzcEOEJFk4D7gdKAMp2fyYlVdHVbsB8AzqvqAiEwFXgCKo4xpyBX29iWwGoExxjuivWvoCRFJBQ5zN61V1cEG5TkeWK+qGwBEZBFwDhCeCBTIctezge3RBh4LPRPUbLPexcYYD4m2Q9lpOHcNbQIEGC8il6vqWwMcVghsDXteBpzQp8wtwCsiciMQBOZFFXWM2AQ1xhgviraN4GfAGao6R1VPBT4P/GKQYyINTNd3jOeLgMdVtQg4C/iN21t5zxOJXCsipSJSWlFREWXI+8c6lRljvCbaROBT1bU9T1T1E8A3yDFlwPiw50XsfennKuAZ95zv4NyZlNf3RKq6UFVLVLUkPz8/ypD3T2HIb43FxhhPiTYRlIrIIyJymrv8Glg6yDHvA1NEZKLbvnAhsLhPmS3A5wBE5EicRBDbn/yD6OlUZhPUGGO8ItpE8DVgFfAN4CacBt/rBjpAVTuBG4CXgTU4dwetEpHbROSLbrFvA9eIyHLgKeArmuBv4MIcm6DGGOMtgzYWu7eBPqKqlwA/35eTq+oLOLeEhm/7Udj6auDkfTlnrIUPR50bTE1wNMYYE3uD1ghUtQvIdy/vjHg9fQmsncAY4xXRdijbBPxLRBYDTT0bVXWfaggHA5ugxhjjNdEmgu3ukgRkuttGZGtqzwQ11qnMGOMV0SaC1ar6+/ANInJ+DOJJOJugxhjjNdHeNfR/o9w2ItgENcYYLxmwRiAiZ+L0+C3sM8poFs4E9iNSQXaAj3eWJzoMY4yJi8EuDW0HSoEvsmcHsgbgm7EKKtEKQjZBjTHGOwZMBKq6HFguIr+LYrTREaNnXoKdda0cMiqY4GiMMSa2om0jOF5EXhWRT0Rkg4hsFJENMY0sgQpzrC+BMcY7or1r6BGcS0FLga7YhTM82AQ1xhgviTYR1KnqizGNZBjpmaDGOpUZY7wg2kTwhoj8FPgTzvzFAKjqBzGJKsFsghpjjJdEmwh6ZhYrCdumwGeHNpzhoyAUsDYCY4wnRDtn8dxYBzLcFIb8fLyzIdFhGGNMzA1415CI/DJs/aY++x6PUUzDgk1QY4zxisFuHz01bP3yPvumD3Esw0pByCaoMcZ4w2CJQPpZH/FsOGpjjFcMlgiSRCRHREaFreeKSC4wosdesAlqjDFeMVhjcTZOJ7Ke2kD47aIj+uJ5T+9iqxEYY0a6wcYaKo5THMNOzwQ1lgiMMSNdtGMN9RKRW2IQx7DTO0GNDTNhjBnh9jkR4AxJ7QmFoQBlViMwxoxw+5MIPHP3UE9fAmOMGcn2JxEcO+RRDFPhE9QYY8xIFVUiEJG7RCRLRHzAqyJSKSKXxDi2hAufoMYYY0aqaGsEZ6hqPXA2UAYcBnw3ZlENE9aXwBjjBdEmAp/7eBbwlKpWxyieYaXAJqgxxnhAtMNQ/0VEPgZagOtFJB8Y8d+ONkGNMcYLoqoRqOoCYDZQ4k5i3wScM9hxIjJfRNaKyHoRWdBPmS+LyGoRWSUiv9uX4GPN77MJaowxI1+0jcXnA52q2iUiPwCeBAoGOSYZuA84E5gKXCQiU/uUmQL8X+BkVT0KuHnf/4TYsglqjDEjXbRtBD9U1QYROQX4PPAE8MAgxxwPrFfVDaraDixi71rENcB9qloDoKrl0YceH4Uhv9UIjDEjWrSJoOdG+i8AD6jqn4HUQY4pBLaGPS9zt4U7DDhMRP4lIu+KyPxIJxKRa0WkVERKKyoqogy5jw1/h4fmQEvtPh1WkO3UCGyCGmPMSBVtItgmIg8BXwZeEJG0KI6N1AO577dpCjAFOA24CHhYREJ7HaS6UFVLVLUkPz8/ypD7COTAjmWw9PF9OswmqDHGjHTRJoIvAy8D81W1Fshl8H4EZcD4sOdFwPYIZf6sqh2quhFYi5MYht646TDxVFjyEHS2R32YTVBjjBnpor1rqBn4FPi8iNwAjFbVVwY57H1giohMFJFU4EJgcZ8yzwFzAUQkD+dS0YZ9iH/fzL4RGrbDqmejPsQ6lRljRrpo7xq6CfgtMNpdnhSRGwc6RlU7gRtwahJrgGdUdZWI3CYiPSOYvgxUichq4A3gu6patX9/ShQmz4P8I+CdeyDKa/49w0xYjcAYM1JF26HsKuAEVW0CEJGfAO8A9wx0kKq+ALzQZ9uPwtYV+Ja7xF5SEsz+Oiy+ETa+BYfOGfSQ3GAqaSk2QY0xZuSKto1A2H3nEO76wTkc9dFfhmA+vHNvVMVFhEKboMYYM4JFWyN4DFgiIj0X188FHolNSDHm88Px18Ib/wUVayH/8EEPKcyxTmXGmJEr2sbinwNXANVADXCFqv4yloHFVMlVkBKIulZgE9QYY0ayQWsEIpIErFDVacAHsQ8pDoKjYOZF8OFv4bM/hIzRAxYvCAUodyeoSUtJjlOQxhgTH4PWCFS1G1guIhPiEE/8nPh16GqH9349aFGboMYYM5JF21g8DlglIq+LyOKeJZaBxVzeZDj8THj/YWhvHrCo9SUwxoxkA14aEpHJwBjg1j675gDbYhVU3My+Ada+AMufguOu6reYTVBjjBnJBqsR/BJoUNW/hy84fQPOjX14MXbISVBwDLx7P3R391vMJqgxxoxkgyWCYlVd0XejqpYCxTGJKJ5EnFpB1Xr45KV+i/l9yeRl2AQ1xpiRabBE4B9gX2AoA0mYqedC9vhBbyUtDPmtjcAYMyINlgjeF5Fr+m4UkauApbEJKc6SU+CE62Dzv2Bb/3fHFoSsL4ExZmQaLBHcDFwhIm+KyM/c5e/A1cBNsQ8vTmZdBmlZA9YKeoaZsAlqjDEjzYCJQFV3qepJOHcNbXKXW1V1tqrujH14ceLPgmMvh1XPQe3WiEUKQgFaOrqotQlqjDEjTLRDTLyhqve4y99iHVRCnHCd03i85MGIuyfmBwF4btnBf9esMcaEi7ZD2ciXXQRH/TssfQJa6/baPWdKPvOOHM1//XUN722sTkCAxhgTG5YIws2+AdobnGTQR1KS8PMLZjIhN53rf7vUhpswxowYlgjCFcyE4s84l4e69m4LyPL7eOjSY2lp7+K6J5fS1tkV4STGGHNwsUTQ1+wboH6b03AcwZQxmfzsyzNYtrWWWxavinNwxhgz9CwR9DXlDBg1ZcB5jedPG8cNcyfz1Htb+d2SLXEO0BhjhpYlgr565jXesRw2/bPfYt88/TDmHJbPjxevZOnmmjgGaIwxQ8sSQSQzLoT0vAE7mCUnCXdfeAzjsgN87cmllNdb47Ex5uBkiSASXwCOu9oZiK7ik36LZaf7WHjZsTS0dnL9bz+gvbP/EUyNMWa4skTQn+OuhuQ0ePe+AYsdMTaLn54/ndLNNdz+/Oo4BWeMMUPHEkF/MvKdS0TLF0FT5YBFz55ewFdPPZTfvLuZZ0ojD1FhjDHDlSWCgcy+ATpbneksB/Hdzx/OKZPz+MFzK1m+tTYOwRljzNCwRDCQ/MPgsPnOBPcdAw9BnZKcxD0XHUN+RhrXPbmUysa2OAVpjDEHxhLBYGbfAM2VziWiQeQEU3no0mOpbmrn67/9gI4uazw2xgx/lggGU3wKjJsB79w34LzGPaYVZnPn/zmaJRur+e8X1sQhQGOMOTAxTQQiMl9E1orIehFZMEC5L4mIikhJLOPZLyIw+0aoWgfrXonqkH8/pogrTi7msX9t4tkPy2IcoDHGHJiYJQIRSQbuA84EpgIXicjUCOUygW8AS2IVywE76lzIKhx0XuNw/++sIzlhYi4L/vgRK7ftPay1McYMF7GsERwPrFfVDaraDiwCzolQ7nbgLmD4ds1N9jkT12z6B7z8/UEbjgF8yUncd/EscoOpfPU3S6luao9DoMYYs+9imQgKgfCb6svcbb1E5BhgvKo+P9CJRORaESkVkdKKioqhjzQaJ3wVjr3CqRU8+BnY+v6gh+RlpPHgJcdS0djGjU99QKc1HhtjhqFYJgKJsK13OE8RSQJ+AXx7sBOp6kJVLVHVkvz8/CEMcR+kpMG//RIufc7pW/DoGfDKD6Fj4IrMjPEh7jh3Gv9aX8VPX14bp2CNMSZ6sUwEZcD4sOdFwPaw55nANOBNEdkEnAgsHpYNxuEmzYWvvQ3HXApv3w0PfQbKSgc85Msl47n0xEN46K0NXPHYe9ZmYIwZVmKZCN4HpojIRBFJBS4EFvfsVNU6Vc1T1WJVLQbeBb6oqgN/qw4H/iz44t1wyZ+gvRkeOR1e/fGAtYMf/dtUvjf/CD7YUsvZ9/yTrz25lE92NcQxaGOMiSxmiUBVO4EbgJeBNcAzqrpKRG4TkS/G6nXjavLn4Pq3YebF8K9fwsI5sG1pxKK+5CS+dtok/vG9udz0uSn8Y10ln//lW9y86EM2VTbFOXBjjNlNtJ9ZuIarkpISLS0dhpWGda/B4huhcRecfBOctsBpV+hHTVM7D721gcff3khHl3L+sUXc+LkpFIYCcQzaGOMVIrJUVSNeerdEMJRaap3bS5c9CflHwrn3Q+GsAQ8pb2jl/jc+7Z3y8qLjx/P1uZMZneWPR8TGGI+wRBBvn7wCf/kGNJbDKd+EOf85YO0AYHttC/f8bT2/L91KcpJw+UnFXDdnErnB1DgFbYwZySwRJEJLDbz0/2D572D0VDj3ASiYOehhm6ua+NVr63h22TbSfclcdcpErvrMoWQHfHEI2hgzUlkiSKRPXobF34CmCqd2MPvrkJ476GHrdjXwy9fW8dePdpDlT+GrcybxlZOKCaalxCFoY8xIY4kg0Vpq4MUFsGIRJPngsM/DjItgyhmQMvCln5Xb6vjFq5/w+sfl5AZTOevoscw/ahwnHJqLL9kGjzXGRMcSwXCx8yNnXoMVz0BTOQRy4egvOVNiFsxyRjrtxwdbanj4Hxt44+MKWjq6CKX7mHfkGOYfNZZTpuTh9yXH8Q8xxhxsLBEMN12d8OnfYPlT8PFfoasN8g53EsL0CyC7sN9DW9q7eGtdBS+v3Mmra3bR0NpJMDWZuUeMZv60sZx2+Ggy7PKRMaYPSwTDWUstrH7OqSlseQcQOHSOc+noyCBwIu0AABPESURBVH+D1GC/h7Z3dvPOhipeWrmTV1fvpLKxndSUJE6dks/8aWOZd+RoQul215ExxhLBwaN6Ayx/2qkp1G4GXxCmnuPUFIo/A0n9twl0dSulm6p5adVOXl65k+11rSQnCbMPHcX8aWM546gxjM60vgnGeJUlgoNNdzdsfddJCKueg7Z6yCqC6efD5HlQWAK+/r/UVZUVZXW8tGonL63cycbKJkRg1oQcTpo0ipLiXGZNCJHpt1tSjfEKSwQHs44Wpx1h+SL49HXQbkhOg/HHO/MpF38Gikr67bCmqqwrb+SllTt5bc0uVm2vp6tbSRI4YmwWx0/MpaQ4h+OKcxljvZmNGbEsEYwULbVOO8Kmfzqzpe1YASik+KHoOCcpFJ8yYGJoautk2dZa3ttYTenmaj7YXEtLRxcA43MDHHdILiXFuRw/MYdJ+RnIAHcyGWMOHpYIRqqWGtjy7sCJYeJnoPDYfhNDR1c3a3bU8/6mGko3VfP+pmoqG51pNXPSfRx7SC7HFedQUpzL0YXZpKZY3wVjDkaWCLyipQY2h9UYdn5Eb2IYfzwccgqMmwFjjoLsooj9FlSVzVXNvLepmtJN1ZRuqmGDO0x2akoSR47NZGpBFlPHZTG1IJsjxmZab2djDgKWCLyqv8QAkJbtJISx05zHMdNg9JERb1etbGyjdFMNH2ypYdX2OlZtr6e2uQNwcsnEUUEnOfQmiCy7Q8mYYcYSgXG0NcCu1bBrJexatfuxvdEtIJB7qJsgjnYTxFEQOmSP2oOqsqOuldXb61m1vZ7VO5zkUFbT0lsmPzOtNykc5SaI4lFBkpKszcGYRLBEYPrX3Q11W5yEsHPl7uRQvYHe2kNq5u6kMPpIyD8c8o+AYP4eCaKupYM1O9zksL2eVdvrWF/eSGe3c5701GSmjM5g0ugMpozOZMroDKaMyaAoJ51kSxDGxJQlArPv2pugfM3uxLDTfWyr210mkOMkhJ7E0POYOa43QbR1drFuVyOrt9ezekc968obWF/eyK76tt7TpKUkMSnfSQpTRmcweXQmU8ZkcEhuOik2sJ4xQ8ISgRkaqtCwEyo+hoq1YY9rnPaIHmlZblLoSRDuEtZAXdfSwfryRtaXN7BuVyPryhtZX97Ittrdl5d8ycKheRlMHpPBZDdRTMwLUjwqaA3UxuwjSwQmtlShqdJJCHskiI+deRh6+IIQmgCh8ZA93nkMTYBsd1twNE0d3Xxa0RiWHBpYV97Ilupmwj+qozPTKM4LMnFU0HnMS6c4L8ghuUECqTYSqzF9DZQI7GeVOXAikJHvLBNP3XNfUxVUrnUuM1V+ArVbnTaJrUugtW7PssmpBLOLmJ49numhCU6SmOEkjLbgVD5ty2ZTTRsbK5vYVNnEpqomXv+4nMrGtj1OMy7bT3F4ghgVZGJekPG56TZctzERWI3AJE5rPdRtdZPDVqjdsufzxl17lpdkyJ3oDNmdf1jvY0PGRDY3JvcmiI1VPYmimeqm9t2HC4zJ9DM+N8D4nHSKctMpynHWx+cGGJcdsEZrM2LZpSFzcOpohfptuxNEzWanVlH5CVR9Ct0du8tmFULeYU67RN5hvet1EmJTdTObqprYWNnE1uoWttY0s62mhR11LXSHffxTkoSCUGCP5DA+N52inHTG5wTIz0yzITfMQcsuDZmDk88PoyY5S19dHVCzyWmLqFwLFZ84jx8+GdYvArL9IWbkH86MnuRwZAFkjIGMItr9eexoS2VrTStba5opq2nuTRSRLjmlpSRRGApQEAr0PhaE/BTmOM/HZvtJS7FLT+bgYzUCM7KoOrWIirVOzSH8sbly7/LJqU5iCOa7CWK0u4yhzZ9HeXcW2zqy2NAaZGM9bK9tZVttC9tqW6hoaNvrdPmZaRT2Jgp/WMJwtoXSfVarMAlhl4aMAecW14ZdTttDU4Xz2LgLGnvWy525pJsqnOG++/IFIXMsZBVA5jg6M8dRn5JHOaPY1hViU3s265oDlNV1st1NFm2de57H70tibJafsdl+9zHA2Kw05zHbz7hsP3kZadZWYYacXRoyBpwOcIEcGH3EwOW6u6C5andyaCzfvd6wHep3wNZ3SWnYSW5XO7lA7xklyalZZI1Di8bR6h9DrS+fcs2lrCvElvYsNrQms7Ghm9LNNeyq30FH154/xpKThNGZaYzJchJDz+PYbD+jM/2MyUpjdJbf5qY2QyamnyQRmQ/8CkgGHlbVO/vs/xZwNdAJVABXqurmWMZkzKCSkndfIhpId7eTMHqSQ89j/XZo2I5UbSDQ8E8CrXWMA2aEH5vih4wx6MSxtAfyaPTlUZOUSzkhdnRms7ktg09bM/hkZytvfdJBU3vXXi+fnprMmCw/+W7SGJ2Z1ptARmemMdpNGJlpKXY5ygwoZolARJKB+4DTgTLgfRFZrKqrw4p9CJSoarOIfA24C7ggVjEZM6SSknb3nxg3o/9y7U27E0XDLmjc6dQwGnYhjTtJq1lPWuO/GNVay+S+x0oyZOXTFRxNa1oejb5c6iSLKs1iV1cm2zuCbG4NsnGLn380+qnt2Lux2u9L6q1J5GemkZeRRn5G2HpmGnmZaeRlpFpjt0fFskZwPLBeVTcAiMgi4BygNxGo6hth5d8FLolhPMYkRmoQ8iY7y0A6Wne3WzTs3GM9uXEXwYadBGvXMqapArra9z4+GTQQpNOfR2tqDo0pOdRLFpWaRXlXJttag2ypz2B1c5CNrUGqyUTZcyyn7ICPvIzUPZLEHgkjmMaojFRGWdIYUWKZCAqBrWHPy4ATBih/FfBipB0ici1wLcCECROGKj5jhhefH3IOcZaBqDpDijdVOJemmiqcIT6aKpDmKnxNFfiaKslsqmRc08cc3lQB3Z17nsMPKsl0+nNpTcujwZdLneRQSYhd3ZmUNWWxqTqDpS1BNrdlUEcQ2PPyUmZaCnmZaYwKprrJIY28oPM4KiOVUUGnljEqI41QwGdDkA9jsUwEkf7VI96iJCKXACXAnEj7VXUhsBCcu4aGKkBjDkoi4M9ylkh9LPpSdYbzaKoIu1uqHGksx9e4C19TBZmNuyho3OzsC++oJzhJI8lHhz+P1tQQzclZNEgmdWRQ1Z1BRUuAHfXpbG/z83arnxrNoFad5NHpfsUkCeQGnaSR25M4gqnOtt51N4kEU8m2xBFXsUwEZcD4sOdFwPa+hURkHvB9YI6q7n1jtjHmwIhAIOQseVMGLqvq3GYbljCcpLGL1KYKUpuryWqpZmzLFmiudspqWEN26p6n60gJ0pqSTVNyFvWSSX1nOjU1Aaoq/FR0+KnoSGOjBmkgQL0GqSedBk2nKSlIaiCT3IwAucFUcjNSyQumEkpPJZTuIyc9lWz3MRRwHjP9KZY89lMsE8H7wBQRmQhsAy4E/iO8gIgcAzwEzFfV8hjGYoyJhgik5zpL/uGDl1eFtnonIfQkhrDF11yNr6WGzJZqxjZXQ9sOp3bSUQ+0gK//U3d3CS0NQZoa0qknSG23n6quILWawS6CrNVManGe15JBHRl0p4WQ9Fz86RmE0lP3TBjpPieRBHzOesDZl5lmCSRmiUBVO0XkBuBlnNtHH1XVVSJyG1CqqouBnwIZwO/d29u2qOoXYxWTMWaIiYA/21lyivft2M52J4m01u25uNuSWusJttYRbKtntLtPW2rQ5m3QUkNSV+ve51SgCTqafNRLJnUEqe7OoLo7nVrNoJIAm/HTrH6aSKMZPy34kbQMktIySPZnkBrIJC2YhT89i2BGFqFgGqF0H9mBPZesgA/fCJk4yXoWG2MOTh0t/dREwp43V0NLLdpSTXdzNbQ3kdTRhETqOd6PJnUSRpP6qSe9twZSqxk0J2fSlpJNR1o2XWkhNJBDUnoOKcFRpGbkkhlM3yNxZAdSyPL7yPT78PuS4tq/w3oWG2NGHl/AWbIKBi0qOJclAOdyVmer07+jvdF9jLze1dpIUnM9qS0NJLc2kNFSS1FrLclt20htryOts4Gkrm5oxllq9nzdBg1Q13P5SoOUk06jBmgkQLOk05GSQZcvg+7UDEjLJMmfRZI/C196Nr70bNIysslITyfLn0JWwMeE3HQKQoGhfR+xRGCM8RqR3UkkmDdg0WQg4C4RdXfvbiPps3Q21ZDSWEl2YxWZzTWMb6lB2mpI7thKSkcjqV1NiCq04yyNkV+iVX00EKBRA6wpvpCCK2/b7z+9P5YIjDFmfyUl7b4ji4l77EphkC9YVaf20dYQttT3rne11tPeVEt7cx3JzXVktNYzbcogd33tJ0sExhiTCCKQluEsjNtr96C1kSE0Mpq8jTHG7DdLBMYY43GWCIwxxuMsERhjjMdZIjDGGI+zRGCMMR5nicAYYzzOEoExxnjcQTfonIhUAPs7wX0eUDmE4Qy14R4fDP8YLb4DY/EdmOEc3yGqmh9px0GXCA6EiJT2N/recDDc44PhH6PFd2AsvgMz3OPrj10aMsYYj7NEYIwxHue1RLAw0QEMYrjHB8M/RovvwFh8B2a4xxeRp9oIjDHG7M1rNQJjjDF9WCIwxhiPG5GJQETmi8haEVkvIgsi7E8Tkafd/UtEpDiOsY0XkTdEZI2IrBKRmyKUOU1E6kRkmbv8KF7xua+/SUQ+cl+7NMJ+EZG73fdvhYjMimNsh4e9L8tEpF5Ebu5TJu7vn4g8KiLlIrIybFuuiLwqIuvcx5x+jr3cLbNORC6PY3w/FZGP3X/DZ0Uk1M+xA34eYhjfLSKyLezf8ax+jh3w/3sM43s6LLZNIrKsn2Nj/v4dMFUdUQvOxD6fAocCqcByYGqfMtcDD7rrFwJPxzG+ccAsdz0T+CRCfKcBzyfwPdwE5A2w/yzgRZw5wU8EliTw33onTkeZhL5/wKnALGBl2La7gAXu+gLgJxGOywU2uI857npOnOI7A0hx138SKb5oPg8xjO8W4DtRfAYG/P8eq/j67P8Z8KNEvX8HuozEGsHxwHpV3aCq7cAi4Jw+Zc4BnnDX/wB8TkQkHsGp6g5V/cBdbwDWAIXxeO0hdA7wv+p4FwiJyN5z7cXe54BPVXV/e5oPGVV9C6juszn8c/YEcG6EQz8PvKqq1apaA7wKzI9HfKr6iqp2uk/fBYqG+nWj1c/7F41o/r8fsIHic787vgw8NdSvGy8jMREUAlvDnpex9xdtbxn3P0IdMCou0YVxL0kdAyyJsHu2iCwXkRdF5Ki4BgYKvCIiS0Xk2gj7o3mP4+FC+v/Pl8j3r8cYVd0Bzg8AYHSEMsPlvbwSp5YXyWCfh1i6wb109Wg/l9aGw/v3GWCXqq7rZ38i37+ojMREEOmXfd97ZKMpE1MikgH8EbhZVev77P4A53LHDOAe4Ll4xgacrKqzgDOBr4vIqX32D4f3LxX4IvD7CLsT/f7ti+HwXn4f6AR+20+RwT4PsfIAMAmYCezAufzSV8LfP+AiBq4NJOr9i9pITARlwPiw50XA9v7KiEgKkM3+VUv3i4j4cJLAb1X1T333q2q9qja66y8APhHJi1d8qrrdfSwHnsWpfoeL5j2OtTOBD1R1V98diX7/wuzquWTmPpZHKJPQ99JtnD4buFjdC9p9RfF5iAlV3aWqXaraDfy6n9dN9PuXApwHPN1fmUS9f/tiJCaC94EpIjLR/dV4IbC4T5nFQM/dGV8C/tbff4Kh5l5PfARYo6o/76fM2J42CxE5HuffqSpO8QVFJLNnHadBcWWfYouBy9y7h04E6nougcRRv7/CEvn+9RH+Obsc+HOEMi8DZ4hIjnvp4wx3W8yJyHzge8AXVbW5nzLRfB5iFV94u9O/9/O60fx/j6V5wMeqWhZpZyLfv32S6NbqWCw4d7V8gnM3wffdbbfhfOAB/DiXFNYD7wGHxjG2U3CqriuAZe5yFnAdcJ1b5gZgFc4dEO8CJ8UxvkPd113uxtDz/oXHJ8B97vv7EVAS53/fdJwv9uywbQl9/3CS0g6gA+dX6lU47U6vA+vcx1y3bAnwcNixV7qfxfXAFXGMbz3O9fWez2HPnXQFwAsDfR7iFN9v3M/XCpwv93F943Of7/X/PR7xudsf7/nchZWN+/t3oIsNMWGMMR43Ei8NGWOM2QeWCIwxxuMsERhjjMdZIjDGGI+zRGCMMR5nicAYl4h09RnZdMhGshSR4vCRK40ZTlISHYAxw0iLqs5MdBDGxJvVCIwZhDue/E9E5D13mexuP0REXncHRXtdRCa428e44/svd5eT3FMli8ivxZmH4hURCbjlvyEiq93zLErQn2k8zBKBMbsF+lwauiBsX72qHg/cC/zS3XYvznDc03EGbLvb3X438Hd1Br2bhdOjFGAKcJ+qHgXUAv/H3b4AOMY9z3Wx+uOM6Y/1LDbGJSKNqpoRYfsm4LOqusEdMHCnqo4SkUqcYQ863O07VDVPRCqAIlVtCztHMc68A1Pc598DfKp6h4i8BDTijJL6nLoD5hkTL1YjMCY62s96f2UiaQtb72J3G90XcMZuOhZY6o5oaUzcWCIwJjoXhD2+466/jTPaJcDFwD/d9deBrwGISLKIZPV3UhFJAsar6hvAfwIhYK9aiTGxZL88jNkt0GcC8pdUtecW0jQRWYLz4+kid9s3gEdF5LtABXCFu/0mYKGIXIXzy/9rOCNXRpIMPCki2Tijuv5CVWuH7C8yJgrWRmDMINw2ghJVrUx0LMbEgl0aMsYYj7MagTHGeJzVCIwxxuMsERhjjMdZIjDGGI+zRGCMMR5nicAYYzzu/wMpCKjofp6iYQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "\n",
    "# Plot history: Loss and accuracy\n",
    "plt.plot(history.history['loss'], label='Train loss')\n",
    "plt.plot(history.history['val_loss'], label='Test loss')\n",
    "#plt.plot(history.history['val_acc'], label='Test accuracy')\n",
    "plt.title('Train and Test Loss')\n",
    "plt.ylabel('Cross-Entropy')\n",
    "plt.xlabel('Epochs')\n",
    "plt.legend(loc=\"upper right\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Train and test accuracy\n",
    "\n",
    "Ideally, as the train performance improves, the test accuracy improves. In general, this does not happen.\n",
    "\n",
    "There are situations when the train and accuracy both improve up to a certain only. Then, the test accuracy degrades while the train accuracy is still improving. This situation is called overfitting. The network is no longer learning but just memorizing the train data. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYgAAAEWCAYAAAB8LwAVAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3deXxU9b34/9c7+76TBMISVgVkEYKIUrciqFWptVas1qVaile81ntr1W+ttda2tr29vW31p6WKaKui1drSWxW1am2vIoRdEGWRJQlLtsk6Wef9++OchCFMwgSYbPN+Ph7zmDnnfM45nzmZzHs+y/l8RFUxxhhjOoro7QwYY4zpmyxAGGOMCcgChDHGmIAsQBhjjAnIAoQxxpiALEAYY4wJyAKE6XNEJFJEakVkeB/Iy79E5MbezocxvcEChDlh7pd528MnIl6/5Wu7ezxVbVXVJFXdG4r8ngwi8oTfe2wSkWa/5b+ewHEXi8jrQaZ9SUQaRCT9eM9nTFcsQJgT5n6ZJ6lqErAXuMxv3bMd04tIVM/n8uRS1Vv83vPPgGf93vNloT6/GxQuBeqABaE+X4dz9/u/nwmOBQgTciLykIi8ICLPi0gNcJ2IzBKRVSLiEZH9IvJrEYl200eJiIpIvrv8B3f7ayJSIyIfiMjITs4V4f6yPuAe+10RGe+3vctjichFIvKJiFSJyK8AOYH3fa6IrHbzsVZEZvltWyQie9w87BSRL4nIDOAXwBy3JFLUxeGvAfa46W/ocN5oEfmBiHwmItVuHga52053r0mle92/5a5/SUTu8TvGpSKyzW+5TET+Q0S2AhXuuh+IyG73PWwWkYv90ouI3O5eyxoR2SQiE9x9nu6Q36dE5KFuX2ATeqpqD3uctAewG5jTYd1DQBNwGc6PknhgBjATiAJGAZ8Ci930UYAC+e7yH4AyoACIBl4A/tDJ+SOAG4FkIA54BCj0297psYBsoBa4wt12F9AC3HiM9/wQsKzDutFAOXCBm6fLgUNAKjAI50t2lJs2DzjVfb0YeD2I6/whcD+QD/iAU/y2/QAodK9rBDDdPW+mm6dFQIy7boa7z0vAPX7HuBTY5rdcBqwCBgPx7roFQC4QCdwEVAMZ7rabgF3AFJwge6r7Pke76RLcdPFAVdv7t0ffelgJwvSUf6nqX1XVp6peVV2jqh+qaouq7gKWAOd2sf9Lqlqoqs3As8DUQInc4y9T1RpVbQAeAKaLSGIQx7oU2KCqr7jbfgGUHuf7vRF4UVXfdvO0AtgOzMH5QhdgoojEqmqxqm7r4lhHEJFTgTOA51R1N/ABcL1fkluAu1V1l3vutapaBXwJ2Kqqj6tqk6pWqeqabryn/1bV/arqBVDV5ap6QJ02o6dwAuDpfnn4kapuVMc2933uBDYC891084FPuvP+Tc+xAGF6yj7/BRE5VUT+5lYFVQMPAlld7H/A73U9kBQokdsD6mcisss97g53k/+xOzvWEP98qqoP6KqapysjgBvd6iWPiHhwAtEQVS3HqRa6EzgoIn8RkdHdOPYNwGpVbXtvzwJfc6vXInF+5e8MsN+wTtYHq+PfcKFbtdT2/vI5fJ27OtfTwHXu6+uA359AnkwIWYAwPaXjsMG/BT4CxqhqCk51yXHX9/u5HrgEp2onFRjjrg/m2PtxvticHUQigKHHmY99wOOqmub3SFTV3wCo6gpVvQCn2qUEpyoMjr5OR3DzdB0wyQ2uB4Afuvk+T1Vb3fcRKODs62Q9OI3dCX7LuQHStOdNRCYA/w3cjFOtlIZTvdh2nbs614vA59xjXAAs7ySd6WUWIExvScape65zG5G/eRKP24hT154A/Kgb+/4vMFVE5rs9de7EaS84HsuAa0TkfPeXfbyIzBGRHBEZJiKXiEg80IDz5dzq7ncQGC6d9xT6vJunqX6PicBfONxY/QTwExHJd889TURSgT8BE9xf/jEikioiBe4+G4DL3HVDcdpCupKEU1VWCkSIyG04JYg2TwD/T0Qmuw3Wp4pIHoCqVgN/wwkMb6nq8VbjmRCzAGF6y3/ifKHV4JQmXjhJx30K5xd5CbAFeD/YHVX1IHA18HOcADMcpzG421R1O3AVTgN2Oc6v69txfmFHAd/FCQZlOF/yd7i7vgoUA6UisifAoW8AXlDVT936/wOqegD4NXCliCS553wD+AfgAf4/IMat2roQpwRSCnwMnOUe93c4vaL2ASuA547x/lYDTwLrca51Hk6QabMM+A3wMk6j9AtAit/2p4FJWPVSnyaqNmGQMaZnudVL7wO5bmcC0wdZCcIY06PchvQ7gWcsOPRtdkekMabHiEg28BlOl995vZwdcwxWxWSMMSYgq2IyxhgT0ICpYsrKytL8/PzezoYxxvQra9euLVPVgN25B0yAyM/Pp7CwsLezYYwx/Uon3akBq2IyxhjTCQsQxhhjArIAYYwxJiALEMYYYwKyAGGMMSYgCxDGGGMCsgBhjDEmoAFzH4QxxgxUrT6l2ttMlbcZj/vc9qj2NpOeEMNXZw4/6ecNaYAQkYuAX+FMav6Eqj7cYfsIYCmHJ3G/TlWL3G2twGY36V5VvTyUeTXGmJ7Q0NxKWW0jZbVNlNY0UlbbSEVdE9XeZjz17pd+fSPN9VWIt4KIRg+xzVWkUUOa1JEuNaTiPA+jjjSpoTp+GMxccdLzGrIA4Q7p+yjOBCVFwBoRWaGqW/2S/RfOkL9Pi8gFwE+Ar7nbvKoacGJ6Y4zpS5pafJTXNVJW00RpbYP73EhpTSOltY2Uuc/lNV5iG8oZKqVHPE4RDxkRtWRIHWlSS7LWEonPObgAMUeeryUmBV9cOhKfTkRiPpGDJ4fkfYWyBHEGsENVdwGIyHJgPuAfICbgjAsP8A7w5xDmxxhjOqWqeJtb23/Ftz1XeZuOWPa41Tqe+mY83iaq6pupbmgBQPCRRRXDpJShUsao6HKmR1cwPKKUXD1EphwkKq75iPO2JgxCknOJSBgCCRkQnwHx6e7r9A7LGRCXSlRkz7QOhPIseTjTF7YpAmZ2SLMRuBKnGuoKIFlEMt2pEeNEpBBoAR5W1aOCh4gsBBYCDB9+8uvfjDH9V0NzK5X1TZTXNlFZ30RFnfuobaSh+hARVUXE1BaT6C0hoqmW5pZmVH1E4kNQIvERgRKBj1h85IkyOkqIi1TiooTYSIiNgrh0SNEa0pv3k+jdT6Sv6ciMxGRB2nBIn+E8pw2HtBHOc+owImMSeucCBSGUAUICrOs4+cS3gUdE5EbgPZy5eFvcbcNVtURERgFvi8hmVd15xMFUlwBLAAoKCmxiC2MGOFWlvK6J/Z4G9ld52V/VQEmVl9KaRirdAOCp8xJdd5D05oPkSdkRj4nuc7w0HX3sSEElApUIIAIiIkEikIhIRJzltmckAiQSRJznuBQYMhXSLne//N0AkDYMYhJ7/kKdJKEMEEXAML/loTiTm7dT1RLgSwDuZOtXqmqV3zZUdZeIvAucDhwRIIwxA4eqUlnfzP7KakrLKimrrKDCU0VNlYfqmirqaqtpqKshxuclQRqJp5EEGhkU0cjU6BqGRpSTq6VktJYSGeGD2MPHbo7NoDVlKBFppxOVMQLSnV/vpA1znuPTEZGAv2rDWSgDxBpgrIiMxCkZLAC+6p9ARLKAClX1Affi9GhCRNKBelVtdNOcDfwshHk1xpxsrS3grcBXW0pNxQFqKw7g9RykqfoQWluGeMuIbqggrrmSuNY6YnwNJNHARGnt/JgBvrE0Kg5JyHK/7Ccc/tJPGwapwyE1j+iYRKJD904HrJAFCFVtEZHFwEqcbq5LVXWLiDwIFKrqCuA84CciojhVTLe5u48HfisiPpyb+R7u0PvJGNNbWhqhqghf5T5qD+2i7tBuWqr2o3VlRHjLiWmqJKG5kiRfDeD8A6e6jzaVmkQlKdRGplIRPRRfQgpRcUnExicTl5hMYnIqyckpJCWnEhGTCDEJEO0+xyQefh2dgERE9sZVCAsDZk7qgoICtQmDjDkJGmvAs4+m8j1UHdhJQ9ketHIvUTVFJDbsJ7mlggi/5kSfCuUkU6EpVLhf+o0xGTTHZkBCJhHJg4hJySYhLYfkzFwysnIZlJpIYqzdp9sXiMhaVS0ItM3+QsaEA1Xni7+uFOrKaK09SG35Aeoq99PsOYBWlxBbV0xK4wES3V/+MTh3sDZpJCWaxW6yqIiaQm3iEFqS84hIG078oHxSc0aQk5HCoORYxiTGEhlhNfkDhQUIY/qz5gao/Axq9kNtKdSV0lp7iAbPQZqrD0JtKZHeMuKaKojWwz13Ijlc7VOlCRzQDPbIIKpiZuNNGYIvZRjRGcNJzB5JRs4whqQnMiM1juhIG74tnFiAMKava2mCyt1QsRPKd+Ir30nTwe1QsZPY+v1Ih97jLRpFFSmUawplmko546iKSKU5LhNNyCYyJZv41BwSM3NJzxpMdnoKeWnxjIuPRsR+/ZvDLEAY0xe0toBnD1TsgvKdaPkOGg9tR8t3EVtbRETbsAtAtSayW3PZpaPY7TuLksghtCYPISolh4S0XNLSMxmcFk9uahx5afFMT40jOTbKvvxNt1mAMCYUVKGpDurLoK7cfS7zey5H68porjmEr7aMmLr9RGhL++51Gs9nmsNnmsdnOp3iiME0pYwkMmsMg3KGMDIrgfzMRGZnJTIoOda+/E1IWIAwJhiq0Fjd/uXe8cv+6OVSaGkIeKhmicZDKqW+JMp8yZQznBI9nX0yhMaUfCIyR5OZM5T8rCTysxKYkZVITnIcEdb4a3qYBQgT3hqqoexTqC7p5Ne+37KvOfAxohPwJWTREJNOdUQqFbF5HIhMYm9DAjvr4yhuSqRCUygnmWpJJTM9g/xBSYzMSiQ/K5GRmYmcMSiRwSkWBEzfYgHChIemOij9BEq3waGtcGib87pq39FpY1MgIRMSsyB1KAyZAglZaEImlZJKcVMin9XH8UlNLJsro/i4vJXSg43tu4vAkNR4RmYlMnKcUw000n3kpcdbTyDTb1iAMANLc4NTIjgiEHwMlXtoHysyMgayxsHwM2HQjZA93gkEiYMgIZNGothTXs+OQ7XsPFTLjtJadu6tZVdpHfVNh4eBSI1XxmTHct64REZnJ5GfmcioQYkMz0ggLtru7jX9nwUI0z+pOtVC+zfA/o1wcIsTFCp2gbo9fiKiIHMMDDkdpnwVsk+F7AmQPhIio6hpaGbHoVq2H6pl565adpYWs+PQJ+ytqMfn13M0Ly2e0dlJzMjPYEx2EqMHJTEmO4nMxBhrHDYDmgUI0/epQlWREwxKNhwOCnWlznaJgIzRTklg4pec5+zxzrqoGKrqm9lRWsP2g7V8urOW7YfWsuNQLfurDjcix0RGkJ+VwIQhKVw2ZUh7IBg1KJGEGPs3MeHJPvmmb1F17gfYv/HIYFBf7myXSBh0KoydC4OnwuApkHsaxCRSUdfE9oM1bD9Uy45dtWw/tI7tB2s5VHO4fSAuOoIx2UmcOSqTMdlJjM12SgPDMxKIsrYBY45gAcL0Hp/PqRI6sOlwINi/EbyVzvaIKKckcMrFTjAYcjrkTKQ5IpadpbVsLalmy6Zqtr6+mU8P1lBed3goicSYSMbkJHPOuEGMzU5ibE4SY7OTyUuLt55CxgTJAoTpGc0NTqPxgU1wYLP7+Aia65ztEdGQMwHGX+7MzDV4CmRPpM4Xxcf7q9m6v5otq6rZun8tnxysoanFaWeIjYrg1MEpzBmfw9gcpzQwNieZIalx1j5gzAmyAGFOvvqKIwPB/k1OzyJ1ewDFJEPuJDj9Ohg8GXJOg+zxHPKqUyooqWbrp9VsLVnF7vI62kakT0+IZuKQVG46K58JQ1KYMDiFkVmJVjVkTIhYgDDHz+dzRhI9uMWvVLAZqosOp0ke4gSB8Zc6QSF3EqTl421RNuzzsG5vJWs3VrK5+J+U+rUVDMuIZ+LgVK44PY+JQ1KYMCSF3BQrFRjTkyxAmODUVzhVRAe3wMGP4OBWZ7m53tkuEc69BSNmuYFgsvOcmAXAoeoGCvdUUvh/lazd8z5bSqppcfuSjs1O4txxg5gw2AkE4wenkBpvE0Qa09ssQJgjtTRB+XYnABz8yA0IW6Cm5HCa+HSnWmjaDU67QfZEpzE5JgEAn0/59FANhZsrWbuniMI9Feyr8AJOm8GUYWksPGcUBfnpTBueTlpCTG+8U2PMMViACGeqULYddrzl9CI6uMUZjqJtzKGIaBh0Coz8HORMdAJBzkRIznXGk3DVN7WwYZ+HtbuLKdxTybq9ldQ0OCOTZiXFUjAinRtm5TN9RDoTh6QSE2VtBsb0BxYgwk1LI+z+F2x/Az593ZmIBpy2gpyJMGaO85wzETLHQtTRv+4bmltZt7eSVTvLeX9nORuLPDS3OtVF43KSuHTyEApGpFOQn87wjARrNzCmn7IAEQ5qDrgBYSXsfMfpWhoVByPPgVmLYdw8SBve6e7NrT42FXl4f0c5H+wqp3BPJU0tPiIEJg1N4+bZo5g5MoNpw9NJTbC2A2MGCgsQA5HPB/vXOwHh05VO9RFASh5M/gqMu8gJDm6bQUetPmVLSRUfuCWENbsr2gepGz84ha+dOYKzRmcyY2QGKXEWEIwZqCxADBQN1bDrHfj0Dae0UHcIEBh2BlzwPSco5Ew8ou2gjc+nfHKwpj0gfPhZeXsbwpjsJL48fSizRmUyc1QmGYnWoGxMuLAA0Z/5fE4D8+rfwq5/OI3Lcakw+vNOQBgzBxIzA+6qqqzf52HFhhL+tnl/+z0IIzITuHTyYM4clcmsUZlkp8T15DsyxvQhFiD6o+YG2PwivP8IlH3iNDCfeavTljBsJkR2Xu3z6cEaVmwo4S8bi9lX4SUmKoILTslmzoQcZo3OJC8tvgffiDGmL7MA0Z/UlUPhk7B6iTPUde4kuGIJTLwiYG+jNkWV9fx1437+sqGYbQdqiBA4e0wWd3x+HHMn5lg7gjEmIAsQ/UHZDlj1KGx4Hlq8zlDXsxY7Dc2ddCEtr23k1c37+cuGEgr3OKOjThuexgOXTeALk4cwKDm2J9+BMaYfsgDRV6nCnvfhg0fgk9ecaTKnXA1n3ubMjBZAbWMLb2w5wF82lPCvHWW0+pRxOUncNe8ULps8hOGZgXstGWNMIBYg+prWFtj6ZycwlKyH+Aw49zsw4xZIyj46uU95e9sh/ry+mLc+Pkhji4+8tHgWnjOK+VOHcGpuSi+8CWPMQGABoq9oqIZ1z8CHj0PVPmcu5S/8N0y5JuD9Ct6mVl5aV8ST/9zF7vJ6MhJjuHrGMC6fMoRpw9NtUhxjzAkLaYAQkYuAXwGRwBOq+nCH7SOApcAgoAK4TlWL3G03APe5SR9S1adDmddeU18B//olrF0GjdUw4my4+GdON9WIo8csKqtt5JkP9vD7D3ZTWd/MlKGpPPrVacydmEO0zYtgjDmJQhYgRCQSeBS4ECgC1ojIClXd6pfsv4BnVPVpEbkA+AnwNRHJAL4PFAAKrHX3rQxVfntcc4Nz/8J7v4CmGpjwRTjrdsibFjD5ztJanvjnZ7y8roimFh9zxuew8JxRzMhPt7GOjDEhEcoSxBnADlXdBSAiy4H5gH+AmADc6b5+B/iz+3oe8KaqVrj7vglcBDwfwvz2DJ8PPnoJ/v5DqNrr9Eia8wNn2OwOVJXCPZUseW8Xb318kOjICK6cNpSbZ49kTHZSL2TeGBNOQhkg8oB9fstFwMwOaTYCV+JUQ10BJItIZif75nU8gYgsBBYCDB/e+WBzfcZn78Eb98H+jc6EOvN/A6POOypZq09ZueUAS97bxYZ9HtISorn9/DF8bVa+dU81xvSYUAaIQPUe2mH528AjInIj8B5QDLQEuS+qugRYAlBQUHDU9j7j0DZ4837YvhJShsIVv4VJXzmqjaG+qYWX1hbxxD8/Y29FPSMyE/jh/Il8efow4mMieynzxphwFcoAUQQM81seCpT4J1DVEuBLACKSBFypqlUiUgSc12Hfd0OY19CoOQDv/BjW/x5ikmDOAzBzEUQfOZxFaU0jz3ywm9+v2oOnvpnTh6fx/y45lQsn5BJpvZGMMb0klAFiDTBWREbilAwWAF/1TyAiWUCFqvqAe3F6NAGsBH4sIunu8lx3e//QWAvv/8Z5tDbCGQvhnO8EHDhv4z4P1z7xIXVNLcyd4DQ8Tx+R0QuZNsaYI4UsQKhqi4gsxvmyjwSWquoWEXkQKFTVFTilhJ+IiOJUMd3m7lshIj/ECTIAD7Y1WPdprS2w4Q9OqaH2IEyYD5//PmSODph8Z2ktNy1bQ1pCNH++7WxreDbG9Cmi2ner7rujoKBACwsLe+fkqs4cDG/eD6XbnBFV5z7kzMXQiQNVDVz52Ps0NLfy0q1nMTIrsQczbIwxDhFZq6oFgbbZndQnqqURln/VmZchYxR85RkYf3mng+gBeOqbuH7ph1R5m1m+8EwLDsaYPskCxIn64FEnOFz4IMy8tctht8EZIuPmpwvZXVbPsptmcFpeag9l1BhjuscCxImoLoH3/gtOvRTOvuOYyZtbfdz23DrW7a3k0a9O46wxWT2QSWOMOT42eM+JeON7oK0w70fHTOrzKXe/vIm3tx3ih/NP45JJg3sgg8YYc/wsQByv3f/nDJlx9h2Qnn/M5A+/vo0/rSvmPy4cx3Vnjgh9/owx5gRZgDgerS3w2ncgdRic/a1jJv/tP3ay5L1dXD9rBLdfMKYHMmiMMSfO2iCOx9qn4OBHcNXTAedq8PfS2iJ+8to2Lp08mAcum2gjrxpj+g0rQXRXXTm8/ZAzH/SE+V0m/fvHB7n75U3MHpPFL74yxSbxMcb0KxYguuvtB6GxxpnUp4vSwJrdFfzbs+uYOCSFx782ndgoG2zPGNO/WIDojpL1sPZpmPlNyB7fabJtB6q5edka8tLieerGGSTFWk2eMab/sQARLFV49TuQmAXn3dNpsn0V9dywdDXxMZE8c/MZZCbZ/A3GmP7JftoGa9MLULQa5j8KcYHvfi6vbeSGpavxNrXyx0VnMTS96wZsY4zpyyxABKOh2hmIL68Apnw1YJLaxhZuWraGkiovf7h5JqfkJvdwJo0x5uSyABGM937mDN99zfNHzQIH0NjSyjd/X8iWkmp+d/10CvJtPgdjTP9nbRDHUvoprHoMTr8O8qYHTHLXHzfxfzvK+dmVk7ng1JwezqAxxoSGBYiuqMLrd0N0Inz+gYBJDlY3sGJjCYvOHc2V04f2bP6MMSaELEB0ZdvfYOfbcP69kDQoYJKiSi8AM0datZIxZmCxANGZZi+svBcGjYcZt3SarNjjBIi89PieypkxxvQIa6TuzP/9Gjx74Ya/QmR0p8lK3AAxODWup3JmjDE9wkoQgXj2wr/+GyZ80RlzqQvFlV5S4qJIjus8iBhjTH9kASKQld8FBOY+dMykJR4veXZDnDFmALIA0dGud+HjFfC5/4S0YcdMXuzxkpdm1UvGmIHHAoS/1mZ47W5nhrizbg9qFydAWAO1MWbgsUZqf6t/B6XbYMHzEH3sUkF1QzM1DS0MsQBhjBmArATRpvYQvPsTGP15OOXioHYpsS6uxpgBzAJEm7d+4Nz7cPFPu5wIyF9bgLAShDFmILIAAVBUCBv+AGfeClljg96t2L2LeqgFCGPMAGQBwueDV++CpFw49zvd2rXY00B0pJBlkwIZYwYga6Su/Ayq9sHcH0Fs9+ZwKPZ4GZwaT0REcFVSxhjTn1iAyBwNt6/rdnAA9yY5q14yxgxQIa1iEpGLROQTEdkhIkdN5Cwiw0XkHRFZLyKbROQSd32+iHhFZIP7eDyU+SQuJeiGaX8lHq81UBtjBqxjliBEZDHwrKpWdufAIhIJPApcCBQBa0Rkhapu9Ut2H/Ciqj4mIhOAV4F8d9tOVZ3anXP2pOZWHwerG6yLqzFmwAqmBJGL8+X+olsiCPan9hnADlXdpapNwHJgfoc0CqS4r1OBkiCP3esOVDXgU2yYDWPMgHXMAKGq9wFjgSeBG4HtIvJjERl9jF3zgH1+y0XuOn8PANeJSBFO6cF/fIuRbtXTP0Tkc4FOICILRaRQRApLS0uP9VZOqvZ5INJsoD5jzMAUVBuEqipwwH20AOnASyLysy52C1TS0A7L1wDLVHUocAnwexGJAPYDw1X1dOA/gOdEJKXDvqjqElUtUNWCQYMCz/gWKodvkrMShDFmYDpmgBCRfxeRtcDPgP8DJqnqrcB04Moudi0C/IdDHcrRVUg3Ay8CqOoHQByQpaqNqlrurl8L7ATGBfWOekjbTXLWSG2MGaiCKUFkAV9S1Xmq+kdVbQZQVR9waRf7rQHGishIEYkBFgArOqTZC3weQETG4wSIUhEZ5DZyIyKjcKq4dnXjfYVcSZWXrKQY4qIjezsrxhgTEsEEiFeBirYFEUkWkZkAqvpxZzupaguwGFgJfIzTW2mLiDwoIpe7yf4T+IaIbASeB250q7POATa5618CFqlqxdFn6T1FldbF1RgzsAVzo9xjwDS/5boA6wJS1VdxAoz/uvv9Xm8Fzg6w38vAy0HkrdeUeLyMy+n+zXXGGNNfBFOCEPdXPdBetRTWd2CrKiWeBitBGGMGtGACxC63oTrafdxBH2sP6GmV9c14m1ttmA1jzIAWTIBYBJwFFOP0TJoJLAxlpvo6mwfCGBMOjllVpKqHcHogGVdR2zwQNsyGMWYAC2Yspjic+xUm4nRDBUBVvx7CfPVpVoIwxoSDYKqYfo8zHtM84B84N7zVhDJTfV2xx0t8dCTpCdG9nRVjjAmZYALEGFX9HlCnqk8DXwAmhTZbfZszzHccwY9baIwx/U8wAaLZffaIyGk4o67mhyxH/YDNA2GMCQfBBIglIpKOM3fDCmAr8NOQ5qqPK/Z4rYHaGDPgddlI7Y6sWu1OFvQeMKpHctWHNTS3UlbbxJBUCxDGmIGtyxKEe9f04h7KS7/Q1oPJZpIzxgx0wVQxvSki3xaRYSKS0fYIec76qBJPA2BdXI0xA18wYyq13e9wm986JUyrm4o99QA2zIYxZsAL5k7qkT2Rkf6i2NNAhEBuqs0kZ4wZ2IK5k/r6QOtV9ZmTn52+r7jSS05KHAe5f9QAABrmSURBVNGRQc3Waowx/VYwVUwz/F7H4cwAtw4IywBh90AYY8JFMFVMt/svi0gqzvAbYamkysvkoWm9nQ1jjAm546knqceZIzrs+HzKfk+DNVAbY8JCMG0Qf8XptQROQJkAvBjKTPVVZbWNNLX6yEuzBmpjzMAXTBvEf/m9bgH2qGpRiPLTpxXZTXLGmDASTIDYC+xX1QYAEYkXkXxV3R3SnPVBNg+EMSacBNMG8UfA57fc6q4LO8XuTHLWBmGMCQfBBIgoVW1qW3Bfx4QuS31XicdLclwUyXE2UZAxZuALJkCUisjlbQsiMh8oC12W+q5ij9dKD8aYsBFMG8Qi4FkRecRdLgIC3l090BVbF1djTBgJ5ka5ncCZIpIEiKqG7XzUJR4vBSPSezsbxhjTI45ZxSQiPxaRNFWtVdUaEUkXkYd6InN9SW1jC1XeZuviaowJG8G0QVysqp62BXd2uUtCl6W+ybq4GmPCTTABIlJEYtsWRCQeiO0i/YBkXVyNMeEmmADxB+DvInKziNwMvAk8HczBReQiEflERHaIyD0Btg8XkXdEZL2IbBKRS/y23evu94mIzAv2DYVKsccChDEmvATTSP0zEdkEzAEEeB0Ycaz9RCQSeBS4EKfn0xoRWaGqW/2S3Qe8qKqPicgE4FUg3329AJgIDAHeEpFxqtravbd38hR7vERHCtnJYVd4MsaEqWBHcz2Aczf1lTjzQXwcxD5nADtUdZd7c91yYH6HNAqkuK9TgRL39Xxguao2qupnwA73eL2mxOMlNzWOiAjpzWwYY0yP6bQEISLjcH7FXwOUAy/gdHM9P8hj5wH7/JaLgJkd0jwAvCEitwOJOKWUtn1Xddg3L8jzhkRxpd0kZ4wJL12VILbhlBYuU9XZqvobnHGYghXop7Z2WL4GWKaqQ3F6Rv1eRCKC3BcRWSgihSJSWFpa2o2sdZ/NJGeMCTddBYgrcaqW3hGR34nI5wn8xd2ZImCY3/JQDlchtbkZd24JVf0AZ0rTrCD3RVWXqGqBqhYMGjSoG1nrnpZWHweq7S5qY0x46TRAqOorqno1cCrwLnAnkCMij4nI3CCOvQYYKyIjRSQGp7pqRYc0e3FKKYjIeJwAUeqmWyAisSIyEmcGu9Xdemcn0YHqBnxqPZiMMeHlmI3Uqlqnqs+q6qU4v+Q3AEd1WQ2wXwuwGFiJ06j9oqpuEZEH/Qb/+0/gGyKyEXgeuFEdW3BKFltxek3d1ps9mEo8DYDdJGeMCS/BDNbXTlUrgN+6j2DSv4rTddV/3f1+r7cCZ3ey74+AH3Unf6FS7KkHbCY5Y0x4Cbaba1hrL0GkWoAwxoQPCxBBKKr0kpkYQ3xMZG9nxRhjeowFiCBYF1djTDiyABEEm0nOGBOOLEAcg6paCcIYE5YsQBxDlbeZ+qZWhqTF9XZWjDGmR1mAOIYidx6IodbF1RgTZixAHIPNJGeMCVcWII7BJgoyxoQrCxDHUOLxEhcdQUZiTG9nxRhjepQFiGModnswidhEQcaY8GIB4hiKPTbMtzEmPFmAOIYSu0nOGBOmLEB0oaG5ldKaRuvBZIwJSxYgunCgyuaBMMaELwsQXbAursaYcGYBogsWIIwx4cwCRBeKK72IQG6qjcNkjAk/FiC6UOLxkp0cS0yUXSZjTPixb74u2DwQxphwZgGiCzYPhDEmnFmA6ITPp5RUNZBnw3wbY8KUBYhOlNU10tTisyomY0zYsgDRiRKPe5NcqgUIY0x4sgDRiWJ3JjmrYjLGhCsLEJ2wmeSMMeHOAkQnij1ekmOjSI2P7u2sGGNMr7AA0Yli6+JqjAlzFiA6UVzptfYHY0xYswDRiZIqL0PSbAwmY0z4CmmAEJGLROQTEdkhIvcE2P5LEdngPj4VEY/ftla/bStCmc+O6hpb8NQ3k5eW0JOnNcaYPiUqVAcWkUjgUeBCoAhYIyIrVHVrWxpVvdMv/e3A6X6H8Krq1FDlryuHezBZCcIYE75CWYI4A9ihqrtUtQlYDszvIv01wPMhzE/QbB4IY4wJbYDIA/b5LRe5644iIiOAkcDbfqvjRKRQRFaJyBc72W+hm6awtLT0ZOX7cICwRmpjTBgLZYCQAOu0k7QLgJdUtdVv3XBVLQC+CvyPiIw+6mCqS1S1QFULBg0adOI5dpV4vERFCNnJVsVkjAlfoQwQRcAwv+WhQEknaRfQoXpJVUvc513AuxzZPhFSxZVeclPjiIwIFOOMMSY8hDJArAHGishIEYnBCQJH9UYSkVOAdOADv3XpIhLrvs4Czga2dtw3VEo8DXaTnDEm7IUsQKhqC7AYWAl8DLyoqltE5EERudwv6TXAclX1r34aDxSKyEbgHeBh/95PoVbs8TLUAoQxJsyFrJsrgKq+CrzaYd39HZYfCLDf+8CkUOatMy2tPg5UWwnCGGPsTuoODtU00upT68FkjAl7FiA6KLZhvo0xBrAAcZSS9pvkrIurMSa8WYDooKjSShDGGAMWII5S4vGSnhBNQkxI2++NMabPswDRQbHH5oEwxhgIcTfX/qjE4yU/M7G3s2FMn9fc3ExRURENDQ29nRUThLi4OIYOHUp0dPDTKFuA8KOqFFd6OXtMVm9nxZg+r6ioiOTkZPLz8xGxYWn6MlWlvLycoqIiRo4cGfR+VsXkp9rbQl1Tqw3zbUwQGhoayMzMtODQD4gImZmZ3S7tWYDwY/NAGNM9Fhz6j+P5W1mA8GM3yRljzGEWIPyUWIAwpl8oLy9n6tSpTJ06ldzcXPLy8tqXm5qagjrGTTfdxCeffBLinPZv1kjtp9jjJSYqgqykmN7OijGmC5mZmWzYsAGABx54gKSkJL797W8fkUZVUVUiIgL/Dn7qqadCns/j1draSmRkZG9nwwKEv2KPl7y0eKtXNaabfvDXLWwtqT6px5wwJIXvXzaxW/vs2LGDL37xi8yePZsPP/yQ//3f/+UHP/gB69atw+v1cvXVV3P//c6A0rNnz+aRRx7htNNOIysri0WLFvHaa6+RkJDAX/7yF7Kzs4849qpVq7jzzjtpaGggISGBZcuWMXbsWFpaWrjrrrt48803iYiIYNGiRfzbv/0bH374Id/61reor68nLi6Od955h+eee46PPvqI//mf/wHgoosu4r777uPMM88kKyuLxYsX88Ybb/CrX/2K119/nVdffRWv18vs2bN57LHHEBE+/fRTFi1aRHl5OZGRkfzpT3/i3nvv5brrruMLX/gCAFdffTU33HADl1xyyQn9DayKyU9xpdcaqI3p57Zu3crNN9/M+vXrycvL4+GHH6awsJCNGzfy5ptvsnXr0VPLVFVVce6557Jx40ZmzZrF0qVLj0ozfvx4/vWvf7F+/Xq+973vcd999wHw2GOPUVJSwsaNG9m0aRMLFiygoaGBBQsW8Oijj7Jx40beeOMNYmNju8x3VVUV06ZNY/Xq1cyaNYs77riDNWvWsHnzZqqqqnj99dcBuOaaa7jzzjvZuHEj77//PtnZ2dxyyy3tJaLKykrWrFnDvHnzTvRSWgnCX4nHy3mnnLy5rY0JF939pR9Ko0ePZsaMGe3Lzz//PE8++SQtLS2UlJSwdetWJkyYcMQ+8fHxXHzxxQBMnz6df/7zn0cd1+PxcP3117Nz584j1r/11lt861vfaq8SysjIYP369QwfPpxp06YBkJqaesx8x8TEcMUVV7Qv//3vf+fnP/85DQ0NlJWVMX36dM4880zKysq47LLLAOfmN4ALLriA22+/nfLycp5//nm+8pWvnJQqKitBuBpbWjlU00heWkJvZ8UYcwISEw+PhLB9+3Z+9atf8fbbb7Np0yYuuuiigPcCxMQcbneMjIykpaXlqDTf/e53mTdvHh999BF//vOf24+jqkdVSwdaBxAVFYXP52tf9s9LfPzh6u36+noWL17MK6+8wqZNm/j617/enjbQcUWEa6+9lueee46nnnqKm266KfDF6SYLEK4DVc7FH2LDfBszYFRXV5OcnExKSgr79+9n5cqVx32sqqoq8vLyAFi2bFn7+rlz5/LYY4/R2toKQEVFBRMnTmTPnj2sW7euPR+tra3k5+ezfv16VJXdu3ezdu3agOfyer1ERESQlZVFTU0NL7/8MgDp6elkZWXx17/+FXACTH19PeD0yvr5z39OXFwcp5xyynG/T38WIFztN8nZQH3GDBjTpk1jwoQJnHbaaXzjG9/g7LPPPu5j3X333dx1111HHeOb3/wmubm5TJ48mSlTpvDiiy8SGxvL888/z6233sqUKVOYO3cujY2NnHvuueTl5TFp0iTuuecepk6dGvBcmZmZ3HDDDZx22mlcccUVzJw5s33bs88+yy9+8QsmT57M7NmzKS0tBWDIkCGMGzfupJUeAERVT9rBelNBQYEWFhYe9/5/LNzHXS9t4h93nccIG6zPmGP6+OOPGT9+fG9nw7jq6uqYNGkSGzduJDk5OWCaQH8zEVmrqgWB0lsJwlXicaqYclOtiskY07+sXLmS8ePHc+edd3YaHI6H9WJyFXvqyU6OJTaq929OMcaY7pg3bx579+496ce1EoSrxNNgQ2wYY4wfCxAum0nOGGOOZAECd6Igj91FbYwx/ixAAOV1TTS1+CxAGGOMHwsQOGMwgQ3zbUx/cTKG+wZYunQpBw4cCGFO+zfrxcTheSCsBGFM/xDMcN/BWLp0KdOmTSM3N/dkZzFoLS0tREX1za/ivpmrHmZTjRpzgl67Bw5sPrnHzJ0EFz/c7d2efvppHn30UZqamjjrrLN45JFH8Pl83HTTTWzYsAFVZeHCheTk5LBhwwauvvpq4uPjWb169RFjMj3++OM8+eSTNDU1MW7cOJ555hni4+M5cOAA3/zmN/nss88QEZYsWcLMmTN56qmn+OUvf4mIMG3aNJ566imuu+46vvzlL/PFL34RgKSkJGpra3nrrbd4+OGHycrKYsuWLWzevJnLLruMkpISGhoauPPOO7nlllsA+Nvf/sb3vvc9WltbycnJ4bXXXuOUU05h9erVZGRk0NraytixYyksLCQjI+PkXHuXBQicAJEYE0lKvF0OY/qzjz76iFdeeYX333+fqKgoFi5cyPLlyxk9ejRlZWVs3uwEMY/HQ1paGr/5zW945JFHAg55cdVVV7Fo0SIA7rnnHpYtW8att97KbbfdxoUXXsjixYtpaWmhvr6ejRs38tOf/pT333+fjIwMKioqjpnXVatWsXXrVoYPHw44gS0jI4P6+noKCgq48soraWxs5NZbb+Wf//wnI0aMoKKigsjISK655hqee+45Fi9ezMqVK5kxY8ZJDw4Q4gAhIhcBvwIigSdU9eEO238JnO8uJgDZqprmbrsBuM/d9pCqPh2qfBZXOl1cbaIgY47TcfzSD4W33nqLNWvWUFDgjBzh9XoZNmwY8+bN45NPPuGOO+7gkksuYe7cucc81qZNm7j//vvxeDzU1NRw6aWXAvDuu++yfPlywBmdNSUlhbfffpurr766/Us6mC/rWbNmtQcHgF/+8pesWLECgKKiInbu3Mm+ffs4//zzGTFixBHHvfnmm7nqqqtYvHgxS5cubS9tnGwhCxAiEgk8ClwIFAFrRGSFqrbP1qGqd/qlvx043X2dAXwfKAAUWOvuWxmKvJZUea2B2pgBQFX5+te/zg9/+MOjtm3atInXXnuNX//617z88sssWbKky2Ndf/31vPbaa5x22mk88cQTrFq1qn3b8Qzv3draesQw4v7Dkr/11lu89957rFq1ivj4eGbPnk1DQ0Onx83Pzyc9PZ133nmH9evXBxXwjkcoezGdAexQ1V2q2gQsB+Z3kf4a4Hn39TzgTVWtcIPCm8BFocqozSRnzMAwZ84cXnzxRcrKygCnt9PevXspLS1FVbnqqqvapyAFSE5OpqamJuCx6urqyM3Npbm5meeee659/fnnn8/jjz8OOF/61dXVzJkzh+XLl7dXLbU95+fntw/p/corr7QPCd5RVVUVGRkZxMfHs2XLFtasWQPA2Wefzdtvv82ePXuOOC44pYhrr72WBQsWdDrv9okKZYDIA/b5LRe5644iIiOAkcDb3dlXRBaKSKGIFLYNedtd9U0tVNY3WwnCmAFg0qRJfP/732fOnDlMnjyZuXPncvDgQfbt28c555zD1KlT+cY3vsGPf/xjwJlD4ZZbbgnYPfbBBx/kjDPO4MILLzxiBrpHHnmElStXMmnSJAoKCti2bRuTJ0/mO9/5Tvs57rrrLsAZCvzNN9/kjDPOYMOGDZ1OO/qFL3yB+vp6pkyZwoMPPtg+vHdOTg6PPfYY8+fPZ8qUKVx77bXt+1xxxRVUVVVx4403nsxLeISQDfctIlcB81T1Fnf5a8AZqnp7gLR3A0PbtonIXUCsqj7kLn8PqFfVX3R2vuMd7ruironvr9jCVwqG8rmxNt2oMcGy4b5716pVq7j33nt55513gt6nu8N9h7KRuggY5rc8FCjpJO0C4LYO+57XYd93T2Le2mUkxvCba04PxaGNMSYkfvSjH7FkyZL2xvJQCWUV0xpgrIiMFJEYnCCwomMiETkFSAc+8Fu9EpgrIukikg7MddcZY0zY++53v8uePXuYNWtWSM8TshKEqraIyGKcL/ZIYKmqbhGRB4FCVW0LFtcAy9WvrktVK0TkhzhBBuBBVT12x2JjTI/qrJeN6XuOpznBphw1xhyXzz77jOTkZDIzMy1I9HGqSnl5OTU1NYwcOfKIbb3VBmGMGcCGDh1KUVERx9uD0PSsuLg4hg4d2q19LEAYY45LdHT0Ub9GzcBiw30bY4wJyAKEMcaYgCxAGGOMCWjA9GISkVJgzwkcIgsoO0nZCQXL34mx/J0Yy9+J6cv5G6GqAYeRGDAB4kSJSGFnXb36AsvfibH8nRjL34np6/nrjFUxGWOMCcgChDHGmIAsQBzW9ewhvc/yd2IsfyfG8ndi+nr+ArI2CGOMMQFZCcIYY0xAFiCMMcYEFFYBQkQuEpFPRGSHiNwTYHusiLzgbv9QRPJ7MG/DROQdEflYRLaIyB0B0pwnIlUissF93N9T+fPLw24R2eye/6jhc8Xxa/cabhKRaT2Yt1P8rs0GEakWkW91SNOj11BElorIIRH5yG9dhoi8KSLb3ef0Tva9wU2zXURu6MH8/VxEtrl/v1dEJK2Tfbv8LIQwfw+ISLHf3/CSTvbt8v89hPl7wS9vu0VkQyf7hvz6nTBVDYsHzpwUO4FRQAywEZjQIc2/AY+7rxcAL/Rg/gYD09zXycCnAfJ3HvC/vXwddwNZXWy/BHgNEOBM4MNe/HsfwLkJqNeuIXAOMA34yG/dz4B73Nf3AD8NsF8GsMt9Tndfp/dQ/uYCUe7rnwbKXzCfhRDm7wHg20H8/bv8fw9V/jps/wVwf29dvxN9hFMJ4gxgh6ruUtUmYDkwv0Oa+cDT7uuXgM9LDw10r6r7VXWd+7oG+BjI64lzn2TzgWfUsQpIE5HBvZCPzwM7VfVE7q4/Yar6HtBxsiv/z9nTwBcD7DoPeFNVK1S1EngTuKgn8qeqb6hqi7u4CmfK317RyfULRjD/7yesq/y53x1fAZ4/2eftKeEUIPKAfX7LRRz9Bdyexv0HqQIyeyR3ftyqrdOBDwNsniUiG0XkNRGZ2KMZcyjwhoisFZGFAbYHc517wgI6/8fs7WuYo6r7wflhAGQHSNNXruPXcUqEgRzrsxBKi90qsKWdVNH1hev3OeCgqm7vZHtvXr+ghFOACFQS6NjHN5g0ISUiScDLwLdUtbrD5nU4VSZTgN8Af+7JvLnOVtVpwMXAbSJyToftfeEaxgCXA38MsLkvXMNg9IXr+F2gBXi2kyTH+iyEymPAaGAqsB+nGqejXr9+ONMpd1V66K3rF7RwChBFwDC/5aFASWdpRCQKSOX4irfHRUSicYLDs6r6p47bVbVaVWvd168C0SKS1VP5c89b4j4fAl7BKcr7C+Y6h9rFwDpVPdhxQ1+4hsDBtmo39/lQgDS9eh3dRvFLgWvVrTDvKIjPQkio6kFVbVVVH/C7Ts7b29cvCvgS8EJnaXrr+nVHOAWINcBYERnp/sJcAKzokGYF0NZb5MvA2539c5xsbn3lk8DHqvrfnaTJbWsTEZEzcP5+5T2RP/eciSKS3PYapzHzow7JVgDXu72ZzgSq2qpTelCnv9x6+xq6/D9nNwB/CZBmJTBXRNLdKpS57rqQE5GLgLuBy1W1vpM0wXwWQpU//zatKzo5bzD/76E0B9imqkWBNvbm9euW3m4l78kHTg+bT3F6N3zXXfcgzj8CQBxOtcQOYDUwqgfzNhunCLwJ2OA+LgEWAYvcNIuBLTg9MlYBZ/Xw9Rvlnnujm4+2a+ifRwEeda/xZqCgh/OYgPOFn+q3rteuIU6g2g804/yqvRmnXevvwHb3OcNNWwA84bfv193P4g7gph7M3w6c+vu2z2Fbz74hwKtdfRZ6KH+/dz9bm3C+9Ad3zJ+7fNT/e0/kz12/rO0z55e2x6/fiT5sqA1jjDEBhVMVkzHGmG6wAGGMMSYgCxDGGGMCsgBhjDEmIAsQxhhjArIAYcwxiEhrh1FiT9rIoCKS7z8SqDF9SVRvZ8CYfsCrqlN7OxPG9DQrQRhznNzx/H8qIqvdxxh3/QgR+bs7mNzfRWS4uz7HnV9ho/s4yz1UpIj8Tpx5QN4QkXg3/b+LyFb3OMt76W2aMGYBwphji+9QxXS137ZqVT0DeAT4H3fdIzhDnk/GGeju1+76XwP/UGegwGk4d9ACjAUeVdWJgAe40l1/D3C6e5xFoXpzxnTG7qQ25hhEpFZVkwKs3w1coKq73IEWD6hqpoiU4Qz/0Oyu36+qWSJSCgxV1Ua/Y+TjzPsw1l2+G4hW1YdE5HWgFmfE2T+rO8igMT3FShDGnBjt5HVnaQJp9HvdyuG2wS/gjGs1HVjrjhBqTI+xAGHMibna7/kD9/X7OKOHAlwL/Mt9/XfgVgARiRSRlM4OKiIRwDBVfQf4DpAGHFWKMSaU7BeJMccW32Hi+ddVta2ra6yIfIjzY+sad92/A0tF5C6gFLjJXX8HsEREbsYpKdyKMxJoIJHAH0QkFWeE3F+qquekvSNjgmBtEMYcJ7cNokBVy3o7L8aEglUxGWOMCchKEMYYYwKyEoQxxpiALEAYY4wJyAKEMcaYgCxAGGOMCcgChDHGmID+fzxnDcK6fS/4AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Plot history: Loss and accuracy\n",
    "plt.plot(history.history['accuracy'], label='Train accuracy')\n",
    "plt.plot(history.history['val_accuracy'], label='Test accuracy')\n",
    "#plt.plot(history.history['val_acc'], label='Test accuracy')\n",
    "plt.title('Train and Test Accuracy')\n",
    "plt.ylabel('Accuracy')\n",
    "plt.xlabel('Epochs')\n",
    "plt.legend(loc=\"lower right\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
